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1 Ville 's Theorem 



Consider the infinite sequences of 0's and l's, often called reals. Some of them are sufficiently 
"disorderly" and "balanced" between 1 and to represent the result of tossing a fair coin re- 
peatedly, each trial independent of the others. The remaining reals look "fixed" in some way, 
not generated randomly. Motivating a pre cise account of this d istinction could elucidate funda- 
mental ideas in probability and statistics. iLi and Vitanvil i 1997b offer a masterful overview of 
work along these lines, the earliest of which appears to be due to Richard von Mises (1919). To 
state his proposal, we introduce some notation. 

Define N = {1,2,3,---}, and let n G N and real q be given. We denote the nth bit in q by 

q(n). The initial finite sequence of length n — 1 in q is denoted by q[n\. That is, q[n] is the initial 

segment of q that precedes q(n). For example, if q = 10101010 • • • then q(l) = 1, q[l] is the 

empty sequence which we denote by e; q[3] = 10 and q(3) = 1. The set of finite sequences over 

{0, 1} is denoted B. A selection function is any map of B into the set {care, don't care}. Given 

a selection function /, the subsequence of q that / cares about is determined by including 

q(n) in the subsequence iff f(q[n]) = care. We use S(g[n]) to denote the sum of the first n — 1 

bits in q. Suppose that the subsequence of q that selection function / cares about is infinite. 

"Thanks to Glenn Shafer for critical comment. Contact: lieb@princeton.edu, osherson@princeton.edu, wein- 
stein@cis.upenn.edu. Postal mail: Lieb, Physics, Princeton University, Princeton NJ 08540. Research supported by 
NSF grant PHY-01 39984- A03 to Lieb. 
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Then we use Sf(q\\n) to denote the sum of the first n bits in this subsequence. In other words, 



k=i 

where ji, j2, • • • are the integers i such that f(q[i\) = care. Of course, the subsequence of q 
that / cares about may be finite or infinite. 

Von Mises' idea was that some countable collection £ of selection functions would justify 
the following definition. 



(1) Definition: A real q is random just in case: 

(a) lim n ^ 00 S(g[n])/n = 1/2; 

(b) for every / G £ , if the subsequence of q that / cares about is infinite then 

lim n ^ 00 S/(g||n)/n = 1/2. 

Intuitively, a random real defeats any strategy of betting a fixed stake on coordinates that 
are chosen by study of preceding bits. But which countable collecti on £ of selection func- 
tions rend ers |(1)| correct, and how could this fact be demonstrated? ILambalgerJ (119871) and 



Li and Vitanvi 



1997L § 1 .9) review the discussion that lasted beyond mid-century. The debate 
included a striking objection to von Mises' definition that was formulated by the French math- 
ematician Jean Ville. He showed that any choice of £ leads Definition |(1)| to declare some 
intuitively non-random reals to be random. Specifically: 



(2) Theorem: (Ville. 119391) Let £ be any countable collection of selection functions. 
Then there is a real q such that: 

(a) lim n ^ 00 S(g[n])/n = 1/2. 

(b) for every / G £ , if the subsequence of q that / cares about is infinite then 

lim n ^ooS f(q\\n)/n = 1/2. 



(c) for all n £ N, S{q[n])/n < 1/2. 



Clause (Q does the damage to von Mises' theory inasmuch as no real q that satisfies 
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for all n 6 N, the number of l's in q[n] does not exceed the number of O's 



appears to be the result of independent, fair coin tosses. Indeed, such a real falls outside of sets 
of measure 1 widely believed to hold the gen uinely random sequences, e.g., those satisfying 
the law of the iterated logarithm (teller! Il95fl p. 157), and even the principle that fluctuations 
should be symmetrical and of order ^/n. 



Ville's proof of|(2)|is arduous, but a more compact argument is given in Uspenskii. Semenov, and Shen 



1990, pp. 174-6) [relying in turn on Loveland ( 1966)]. We here exploit the combinatorial trick 



introduced in the latter paper but for a somewhat different construction (perhaps easier to fol- 
low). Both proofs strengthen Ville's original result by showing that each selection function in £ 
that cares about an infinite subsequence of the constructed q behaves too regularly; see Section 

We conclude this section with some more notation. Infinite sequences (over any set of ob- 
jects) are assumed to be ordered like N. Given such an infinite sequence 7, we interpret 7(71) 
and 7[n] respectively as the contents of the nth position in 7 and the initial sequence of length 
n — 1 in 7 (just as for reals). A tail of an infinite sequence 7 is any subsequence of 7 that 
excludes just a finite initial segment. Given two finite sequences r, a over any set of objects, 
the concatenation of r to the end of a is denoted err. We'll make use of the following example. 

(3) Example: One selection function, h, satisfies: 
h{a) = care for all a G B. 

Thus, for all reals q, the subsequence of q that h cares about is all of q. 
2 Intuitive motivation for the proof 

We attempt to convey the underlying idea of our proof of Theorem |(2)| Subsequent develop- 
ments are self-contained, so the present section may be skipped. Let us first consider a weaker 
version of Ville's theorem, in which £ is finite. 

See [ La mbalgenl fl 987 , 1996) for proofs of versions of the theorem, relying on probabilistic constructions. 
iLambalgenl Il99o) also discusses whether Ville's theorem is as devastating to von Mises' program as generally 
believed. 
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(4) Finite version of Ville's Theorem: Let £ be any finite collection of selection 
functions. Then there is a real q such that: 

(a) ]hn Vr -> <x S(g[n])/n = 1/2. 

(b) for every / € £ , if the subsequence of q that / cares about is infinite then 

lim ri ^ 00 S/(g||n)/n = 1/2. 

(c) for all n € N, S{q[n])/n < 1/2. 

To prove |(4)| we shall assume that h of Example |(3)| is a member of £. Then it suffices 
to construct a real q that satisfies clauses © and ©. We construct the desired q in stages, 
q(l), q(2), At each stage n, we also define the subset C(n) of £ that cares about q[n]. 

Stage n: Suppose that C(m) for all m < n and q[n] have been defined. Set 

C(n) = {/ G £ : f{q[n}) = care}. Set q{n) = card{j < n : C(j) = C(n)} 
mod 2. 

In words, we set the bit g(n) to zero if the subset of £ that cares about the initial segment of 
length n — 1 [namely, {/ G £:/((/ [n]) = care}] appears an even number of times earlier in 
the construction; otherwise, we set q(n) to one. It is obvious that q satisfies |(4]fa since every 1 
appearing in q is preceded by an occurrence of that can be uniquely chosen to match it. 

Let / e £ be given with {n : f(q[n]) = care} infinite. (If there are no such / in £, we 
are done.) Let ni,rt2, ... be an increasing enumeration of {n : f(q[n]) = care}. Then B = 
C(ni), C(n2), ■ ■ ■ contains exactly the members of the sequence C that include /, in particular, 
no set appearing in B also appears outside of B. Hence, for all m 6 N, the value of q(n m ) 
depends on just B. Subsets of £ that occur only finitely often in B ultimately stop occurring 
altogether since there are only finitely many of them. Therefore, the number of l's and O's in 
q[n m ] is ultimately governed by the subsets of £ that occur infinitely often in B. The latter 
collection is nonempty because B is infinite and there are only finitely many distinct subsets of 
£ that contain / (so at least one of them must occur infinitely often in B). Observe also that for 
k = card(£), no more than 2 k zeros can occur consecutively in q since a block of zeros requires 
that different subsets of £ care about each coordinate in the block. The construction of q now 
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makes it evident that 



cardjj : q(rij) = 1 and j < m} 1 
m 2 ' 



lim 



demonstrating |(4^|b ] and finishing the proof of the finite version of Ville's theorem. Indeed, our 
construction proves a little more inasmuch as it guarantees that for every selection function / 
with {n : f(q[n\) = care} infinite, 

< ^ - S f (q\\n) < 2 card ^ for all n. 



How can we extend this reasoning to Theorem (2) ? We can't consider subsets of an infinite 
collection of selection functions since each might occur just once in the sequence C. This would 
make q into a sequence of zeros. The next idea might be to enumerate £ as /i , /2, . . ., then carry 
out the foregoing construction with {/j : i < n} for increasing values of n. In other words, we 
would build a real q as in the finite case for {/i} but stop at q[k{\ for k\ large enough to ensure 
that S j x (q\\n)/n is at least 1/4, where n is the number of bits in q[ki] that /i cares about. Then 
we would continue to build q starting at q[ki] but this time on the basis of /2}- We would 
stop at q[k2\ for &2 > k\ large enough to ensure that both S ^(qWm) / 'm and Sf 2 (q\\n)/n are at 
least 3/8, where m and n are the numbers of bits in q\k%\ that f\ and fi care about, respectively. 
And so forth. 

This seductive plan is foiled, however, by the prospect that /2, for example, will cease to care 

about q prematurely during the second stage, making it impossible to ensure that S/ 2 (q\\n)/n > 

3/8. Yet if we continue the construction despite this setback, there is no guarantee that /2 will 

care only finitely often in q overall rendering its behavior irrelevant. Indeed, /2 might care 

exactly once in stage 3, perhaps at the same initial segment as f%, then care exactly once in 

stage 4, perhaps at the same initial segment as f^, and so forth. In the end, f'2 may care infinitely 

often but almost always in the context of a unique set of other selection functions. In this case, 

C{k) will be a new subset of £ for cofinitely many k among {j : f2(q[j]) = care}. In turn, 

q(k) will be set to zero for a cofinite subset of the coordinates where f2 cares. 2 

2 Another approach is to attempt to map each selection function / into another /' such that for all reals q, 
{i : f (q[i\) = care} is infinite, and {i : (q[i\) = care} = {i : f(q[i]) = care} if the latter set is infinite. It can 
be shown, however, that there is no such mapping. Hint: Consider the selection function that cares about a 6 B iff 
1 appears somewhere in a (i.e., a is not a block of O's). 
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Our proof of Ville's Theorem extends the construction for the finite case but uses a combi- 
natoric trick to avoid the difficulty just described. At stage n of the construction of q we build 
a finite subset C(n) of £ that is used to determine q(n) as in the finite case (by determining the 
parity of the set of its previous co-occurrences in the construction). The rule for constructing 
the sequence C, however, does not allow f\ to appear with fk+m+i untl l it has appeared suffi- 
ciently often by itself or with some of f\ . . . fk+ m - By defining "sufficiently often" in the right 
way, this maneuver builds up enough parity reversals to ensure that lim n ^ooS / fc (q\\ n)/n = 1/2 
if the subsequence of q that cares about is infinite. 

To make all this clear, it will be notationally simpler to work with just the indexes of our se- 
lection functions. We start by presenting the combinatorial core of the argument before turning 
to its application to Ville's Theorem. 

3 A combinatorial construction 

Let A be the class of infinite sequences of subsets of N that contain 1; that is, for A G A and 
i G N, A(i) C N and 1 G A(i). We define a map * from A into itself. We denote the result 
of applying the map to A G A by A*. For A G A, each coordinate of A* will be a nonempty, 
finite subset of the corresponding coordinate of A. To describe * let A G A be given. A* (n) 
will be the subset of A(n) consisting of the numbers in A{n) that are less than or equal to a 
certain number I(n) which, in turn, will be determined by A[n\. 

Stage n of the construction of A*: We suppose that for all m < n, A*{m) and I(m) have 
been constructed with 

A*(m) = {j G A(m) : 1 < j < I(m)}. 

Then we define: 

I(n) = mini(3j G A(n) : cardjm < n : j G A*(m) and I(m) = i} < 3*) 
A*(n) = {j G A(n) : 1 < j < I(n)} 

Note that 1(1) = 1 and A*(l) = {1}. Evidentally: 

(6) The construction of A*(n) depends on just {A(i) : i < n}. 
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It is also easy to see that: 

(7) For all iGN, I(n) = i for only finitely many n (indeed, for at most i ■ 3* many n). 

Now fix I G N and suppose that it occurs infinitely often in A (for example, I might be 1). 
Let {n : I G A(n)} be enumerated in increasing order as n\, ri2, ■ ■ ■ ■ Then by |(7)[ 

(8) For cofinitely many m G N, t G A*(n m ). 

Now we consider the sequence of integers £ = I{n\), I(n%), • • • . It follows at once from |(5)| 
that: 

(9) for all k > i, there are at least 3 k many occurrences of k in £ prior to the first occurrence 
of k + 1 in C 

For k > £, define: 

a(k) = A*(n m ),A*(n m+1 ),--- ,A*(n m+r ) 

where n m is the first occurrence of k in £, and n m+r+ i is the first occurrence of k + 1 in £. 
From |(8)| and |(9)| we have: 

(10) There is k > t and tail t of A*(ni), A*(n 2 ), ■ ■ ■ such that: 

(a) t has the form a(k) a(k + 1) a(k + 2) 

(b) I is a member of every coordinate of i. 

Specifically, k can be chosen to be the first occurrence of a number in £ such that all later 
numbers occurring in (, are greater than t. Now fix some k and t as described in |(10)| (We leave 
implicit the dependence of k and t on I.) By the definition of n\, n% ■ ■ ■ , we have: 

(11) For cofinitely many members m of {n : £ G A(n)}, A*(m) appears in t. 

From the definition of a(i), for all i > k, each of the sets appearing in a{i) is a subset of 
{1 • • ■ i} so there are at most 2 i of them. Along with |(9)| this yields: 

(12) t has the form a(k) a(k + 1) a(k + 2) • • • , where for all m > 0, a(k + m) has length 
at least 3 fc+m and contains at most 2 k+m distinct sets. 
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4 From finite sets to bits 



Recall that we have fixed A £ A, and thus also fixed A*. We describe a method for mapping 
A* into a real q. For n G N, the preceding parity of A*(n) in A* denotes: 

card{j < n : ^4*(j) = A*(n)} mod 2. 

That is, the preceding parity of A*(n) in A* is if A*(n) appears earlier in A* an even number 
times; it is 1 if it appears an odd number of times. The real q is now defined as follows. For all 
n £ N, q(n) is the preceding parity of A*(n) in A*. 

Let n G N be given, and consider 

B = {i<n: q(i) = 0} 

B 1 ={i<n: q(i) = 1}. 

The construction of q implies that each member of B\ can be paired with a unique, smaller 
member of Bq. Therefore: 

(13) For all n G N, S(q[n])/n < 1/2. 

Recall that we also fixed I G N that occurs in infinitely many coordinates of A. As before, let 
{n : i G A(n)} be enumerated in increasing order as ni,ri2, • • • . Let g denote q{n\), q(n2) • ■ ■ 
We wish to demonstrate that: 

(14) lim^ooS (q[n])/n = 1/2. 

For this purpose it suffices to exhibit a tail s of q that: 

(15) lim^ooS^nD/n = 1/2. 



To specify s, let t be the tail of A*(ni), A* (112), ■ ■ ■ described in (12) We define s to be such 



that s(l) = q(n m ) iff t(l) = A*(n m ). [That is, s excludes an initial segment of q equal in 
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length to the initial segment of A*(ni), A* (712), • • • excluded by t.] We now show that this s 



conforms to (15) 



Recall from |(10)| that t has the form a(k) a(k + 1) a(k + 2) • • • , and is such that for all 
i G N, £ 6 t(i). Let j > be given, thought of as a coordinate of t and also of s. Without loss 
of generality, we assume that j is big enough so that there is m(j) such that t(j) falls within 
a(k + m(j) + 1). We define 

iVo(j) = the number of O's in s[j], and 
Ni(j) = the number of l's in s[j). 



There follow some properties of No(j) and N±(j) which are consequences of (12) and the fact 
that t is composed of all and only the sets of A* that contain £, except for a finite "head." [The 
preceding parity of t(j) in A* therefore depends on just the preceding members of t.] 

First, since the block a(k + m) has at least 3 fc + m coordinates, we have: 

(16) N {j) + N x {j) >3 k+m( -i\ 

From |(12)1 there are at most 2 k+l distinct sets in a(k + i), and this number bounds the number 
of unmatched O's. So: 

m(j)+l 

(17) N (j)<N 1 (j)+ £ 2 k+i <N l { ] ) + 2 k+m ^+ 2 . 



From |(1 7)| we infer: 

(18) Ni(j) > I (N {j) + NtU) ~ 2 k+m ^+ 2 ) . 
Let p be the length of the "head" missing from s. Then: 

(19) N^j) <N (j)+p. 

This inequality allows for the presence of unmatched O's in the head, which would induce 



unmatched l's afterwards. Similarly to the transition from (17) to (18) we see fhat|(19)|implies: 
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(20) tfia)<i(Wo(j)+JVi(j)+p)- 



We now evaluate R(j) = N\ (j) / (Nq (J) + Ni (j). Because we've neglected only finitely many 
terms [that is, R(j) for j with t(j) a coordinate of a(k)], it is clear that if linij^oo R(j) = 1/2 
then (15) is true. For an upper bound, we use (20) | and compute: 



Nati) + Ni(j)+p 



2(N Q (j) + Nt(j)) 

which goes to 1/2 as j goes to infinity. For the lower bound, we use |(18)| and calculate: 

NqU) + JViO') - 2 k+m( ^ +2 1 2 fc+m (j') +1 



m > 



2(iVo(j)+iV 1 (j)) 



and this also converges to 1/2 in view of (16) 



5 Application to Ville's theorem 

To return to Ville's Theorem |(2)| without loss of generality we may assume that £ can be enu- 
merated without repetition as /i, fi ■ ■ ■ where f\ is the "always care" function of Example |(3)| 
For, it's clear that if |(2)| holds for £ ' D £ then it holds for £. So, in the preceding construction, 
we may conceive of the members of A(i) — the coordinates of the infinite sequence of subsets 
of N — as indexes for selection functions in £. Our goal is to construct a real q = q(l) , q(2) , . . . 
with the properties stated in Theorem |(2)| Because the "always care" function appears in £ , it 
suffices to demonstrate |(21blc1 

The construction is built on the results of the previous sections. There, we were given an 
infinite sequence A(l) , A(2) , . . . of subsets of N and these were reduced, by our construction, 
to an infinite sequence A*(1),A*(2), ... of finite subsets of N. [In fact, A*{n) C A(n) for all 
n.] Finally, we showed how to map A* into a real q(l), q{2), .... 

We note that the value of q(n) depends only on Afn + 1] = {A(l), A(2), . . . , A(n)}. There- 
fore, all we have to do for Ville's theorem is to start with A(l) = {m £ N : f m (e) = care}, 
and produce q(l) on the basis of A*(l). [It's easy to see that q(l) = 0.] Next we define 
A(2) = {m £ N : f m (q(l)) = care}, and produce q{2) from A*(1),A*(2). Similarly, A(3) is 
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the subset of N consisting of the subscripts of all selection functions that care about the finite 
sequence q(l), q{2), and so on, ad infinitum. 

The real q that witnesses Ville's theorem has now been constructed. The bounds |(T8) (19) 



describe the number of l's and O's that appear in the subsequence of q about which ft "cares." 
This concludes the proof of Ville's theorem in its original formulation. In other words, we have 
constructed a binary sequence with the property that the entire sequence has a running sum 
Si(n) that never exceeds n/2 and yet each selection function ft that cares infinitely often has a 
ratio Si(n)/n that converges to 1/2 as re — > oo. But much more can be learned from |(T8)l|(T9) 
that were not previously noted, as far as we are aware. 



6 Improvements to Ville's Theorem 

Let q be the real constructed by the method described above. Choose a selection function ft 
that "cares" about q infinitely often (e.g., f\). We define the fluctuation (or fluctuation about 
the mean) for selection function ft to be 



S e (n) = S fi (q\\n)-n/2. 

From |(19)1 we learn that &i is bounded above by an i'-dependent constant. This property mimics 
the behavior of the fluctuation for the entire q sequence (i.e., for /i), whose fluctuation is never 
positive. 



For a bound in the other direction, we can use |(16)| and |(18)| to conclude that there is a number 
Co > such that for all n 



(21) S e (n) > -Qre ln2 / ln3 . 



A quick look at our proof, however, shows that the appearance of In 3 in (21) comes from our 
use of 3* in the definition |(5)| of /(re). We could have used r % instead, as long as r > 2, notably, 
r = 2 x l e with e < 1. By replacing the number 3 by r in the preceding sections, and making no 
other changes, we conclude that for every e > 0, there is a constant Cf (e) > such that: 

(22) for every n, 5t(n) > — Ct{e) n e . 
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The existence of an n-independent upper bound is not affected by this change of 3* to r\ 



The bound |(22)1 is indeed remarkable. For random coin tosses the law of the iterated loga- 
rithm states th at the fluctuat ions exceed (1 — e')Vn In \nn/y/2 (for any e' > 0) infinitely often 
almost surely (Feller, 1950). Our fluctuations are absolute, not probabilistic, and suggest that a 
more clever strategy would reduce the fluctuations even further. Indeed, it is easy to see that for 
any slow-growing function g, for example Inn, there is a suitably fast-growing function h, so 
that our construction with h(i) in place of 3* will enforce a bound analogous to |(22)| with g(n) 
in place of n e and a constant Ce(g) in place of Ce(e). 
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